Image analysis platform for identifying artifacts in samples and laboratory consumables

ABSTRACT

A High-resolution Image Acquisition and Processing Instrument (HIAPI) performs at least five simultaneous measurements in a noninvasive fashion, namely: (a) determining the volume of a liquid sample in well (or microtubes) containing liquid sample, (b) detection of precipitate, objects or artifacts within microliter plate wells, (c) classification of colored samples in microliter plate wells or microtubes; (dl determination of contaminant (e.g. wafer concentration}; (e) air bubbles; (f) problems with the actual plate. Remediation of contaminant is also possible.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a §371 national stage entry of InternationalApplication No. PCT/US2009/064565, filed Nov. 16, 2009, which claims thepriority of U.S. provisional patent application No. 61/114,821, filedNov. 14, 2008, which is incorporated herein by reference in itsentirety.

FIELD OF THE INVENTION

Embodiments of the invention are directed to high resolution imageacquisition, spectroscopy and processing instruments and methods ofanalyzing the quality of a sample and laboratory consumable.

BACKGROUND

The vast majority of drug and probe discovery institutions possess large(10⁵ to 10⁶ member) compound libraries for high-throughput screening(HTS) purposes. Since the cost of a screening library compound averagesbetween $10-100/mg, most institutions have implemented a compoundmanagement (CM) paradigm to efficiently steward this valuable asset. CMparadigms vary from institute to institute, but in general thestewardship of HTS compound libraries is shared between CM and HTSdepartment staff.

HTS operations have benefited greatly from the availability ofrelatively inexpensive and standardized plastic consumables for storageof compound libraries. Currently, Polypropylene (PP) microtube racks(MTRs) and PP microtiter plates (MTPs) are the most popular consumablesfor storage of HTS compound libraries. Copies of compound librariesdestined HTS efforts are typically distributed in a variety of formatsand containers, but two paradigms currently predominate. The mostpopular is the “daughter plate” paradigm. In this case, a large aliquotof the compound library (approximately 10-100 microliters per compounddissolved in DMSO) is removed from the compound stocks (“mother plates”)and distributed to the FITS operation in “daughter” microtiter plates(MTPs). As screeners execute multiple FITS campaigns, successivenanoliter to microliter aliquots are removed from the daughter MTP andadded to HTS assay MTPs. In this scenario, a daughter plate is expectedto last anywhere from several months to years. Compound libraries canalso be distributed, in an “assay ready” format, where small volumes ofeach library compound (approximately nanoliters to microliters ofcompound dissolved in DMSO) are transferred directly to anHTS-compatible assay MTP; assay reagents are then added to this sameplate as part of the HTS protocol, and the entire plate is discardedupon completion of the assay.

The evolution of reliable HTS automation and acceptance of HTSmethodology has caused the size of institutional screening libraries toballoon. As a compound is solvated, formatted into a storage containerand distributed to scientists for testing, it is subject to theintroduction of artifacts. Since most institutions use similar equipmentand procedures to manage and store their HTS compound libraries, a largebody of relevant research has been conducted on where an artifact isintroduced into compound libraries (Kozikowski et al. 2001 The effect offreeze/thaw cycles on the stability of compounds in DMSO. In 7th AnnualConference of the Society of Biomolecular Screening Baltimore;Kozikowski et al. J Biomol Screen, 8, 210-5, 2003; Cheng et al. J BiomolScreen, 8, 292-304, 2003; Bowes et al. J Biomol Screen, 2006). The mostcommon artifact stems from the use of DMSO to dissolve HTS compoundlibraries. Although it is well known as a “universal” solvent, it isalso hygroscopic. The presence of water in HTS compound libraries has adeleterious effect the solubility (and stability) of certain compounds.This source of compound precipitation (and degradation) is exacerbatedby repeated freeze-thaw cycles, or by storage of compounds for extendedperiods of time in uncontrolled atmospheres, empty versus frill wellse.g. on a screening platform during an HTS campaign.

SUMMARY

This Summary is provided to present a summary of the invention tobriefly indicate the nature and substance of the invention. It issubmitted with the understanding that it will not be used to interpretor limit the scope or meaning of the claims.

The systems provided herein address an unmet need in high throughputscreening (HTS) compound library management quality assurance (QA) andquality control (QC). The systems perform at least six simultaneousmeasurements and/or analysis of results in a noninvasive fashion,namely: (a) determining the volume of a liquid sample in wells (ormicrotubes) containing liquid sample, (b) detection of precipitate,objects or artifacts within microtiter plate wells, (c) classificationof colored samples in microtiter plate wells or microtubes; (d)determination of water concentration (i.e. contaminant); (e) airbubbles; (f) problems with the actual plate. It also may employ methodsto remediate certain artifact, such as the contaminant described in (d)above.

In a preferred embodiment, the system comprises an full spectrum,ultraviolet, visible, infrared or near infra red light source forirradiating a consumable or consumable comprising a sample; at least onelens in optical alignment with light transmitted through or reflectedfrom the sample; at least one sensor for detecting reflected light fromthe sample and/or light transmitted through the sample; a computer orprocessor running software for processing and analyzing the reflected ortransmitted light.

In another preferred embodiment, the system optionally comprises atleast one filter in optical alignment with the light source and/orsensor for receiving light emitted from the light source, or transmittedthrough or reflected from the sample.

In another preferred embodiment, at least one sensor is in opticalalignment with at least one lens for focusing light reflected from ortransmitted through the consumable or consumable comprising a sample.

In another preferred embodiment, the sensor is an image sensor andcaptures an image of the reflected or transmitted light from theconsumable or consumable comprising a sample.

In another preferred embodiment, the captured images are standardizedprior to analysis by the software.

In yet another preferred embodiment, the standardization of images bythe software comprises at least one step of: selecting an area ofinterest for analysis; compensating for environmental artifacts;comparing each pixel value to an assigned threshold value; identifyingand enhancing objects; and segmenting the image. In one aspect, a Bayerfilter spectral profile analysis of color space (RGB, HTS, CIE) data isconducted. Preferably, the standardized image is analyzed and resultsstored, in another aspect, only light within a specified bandpass isanalyzed.

In another preferred embodiment, the system comprises a user interlace.In a preferred embodiment, the user interface is a graphical userinterface. In another embodiment, the user interface is combined withone or more databases to store, query, and retrieve relevant sampleinformation, for example.

In another preferred embodiment, the sensor comprises a charge-coupleddevice (CCD), visible, UV, Near-Infra Red (NIR), or FPA NIR camera, or asingle sensor capable of measuring more than one region of the lightspectrum.

In another preferred embodiment, a method of detecting and analyzingartifacts in a consumable or consumable comprising a sample, comprisesthe steps of irradiating the sample with ultraviolet, visible, infra redor near infra red light, or a bandwidth of interest; capturing an imageof light reflected from the sample or transmitted through the sample;and, processing and analyzing the image.

In another preferred embodiment, the software standardizes the imageprior to analysis, said standardizing comprising at least one step of:selecting an area of interest for analysis; compensating forenvironmental artifacts; comparing each pixel value to an assignedthreshold value; identifying and enhancing objects; and, segmenting theimage.

In another preferred embodiment, analyzing comprises measuring at leastone of consumable quality; sample volume, quantitative determination ofDMSO and water quantities; precipitation, color classification, orcontaminant remediation.

Other aspects are described infra.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a scan of a photograph showing an embodiment of theHigh-Resolution Image Acquisition and Processing Instrument (HIAPI). Inthis embodiment, the HIAPI is configured for visible light illumination“bottom read” mode. The consumable, (e.g. microtiter plate) is placed onthe stage (120), an image is captured by the graphical user interface(GUI) software which then displays the results. Analysis is completed inabout minute for a 384-well plate.

FIG. 2 is a scan of a photograph showing a page of the graphical userinterface (GUI). Through the selection of one of the radio buttons, thedifferent functionalities of the HIAPI can be accessed.

FIG. 3 is a schematic illustration of HIAPI image acquisition,standardization and analysis process. Once an image is captured by theHIAPI's optical subsystem, the image is passed to the HIAPI's computermemory for processing and analysis. Data retrieval for offline analysisis also illustrated.

FIG. 4 shows examples of MTP well types from a Greiner 384-wpolypropylene plate. (Left to Right) A normal DMSO filled well, partialfilled well, colored compound, precipitated compound, and an empty wellare analyzed using the MATLAB software applications.

FIG. 5A is a graph showing an NIR spectral overlay of 70-100% DMSOsamples. FIG. 5B is a graph showing, a quantitative DMSO analysis at1.18 microns, FIG. 5C is a scan of a photograph showing NIR imaging ofDMSO (false colors) filled wells within a 384 polypropylene plate, at 0μL, 20 μL, 40 μL, 60 μL, and 80 μL of volume. FIG. 5D shows a linearregression and fit to the well data using the central 6,400 pixels. FIG.5E is a scan of a photograph showing the NIR imaging of water in DMSO(false colors) using a 384 polypropylene plate (empty wells shown inorange). Six wells from top left to bottom right were tilled with 0%,5%, 10%, 15%, 20% and 25% water (w/w) in DMSO. FIG. 5F shows a secondorder regression and fit to the AOI (6,400 central pixels) of each well.FIG. 5G shows a NIR spectral overlay of 0-30% Water/DMSO. FIG. 5H showsquantitative water analysis by NIR at 1.55 microns. FIG. 5I is a plotshowing NIR results from a polypropylene plate at 1.2 mm thickness. FIG.5J is a plot showing NIR results from 100 mM acetaminophen in 75% DMSOand 75% DMSO alone.

FIG. 6A is a scan of a photograph showing: Top-six wells were imagedusing visible light (left) and NIR light (right). The wells contents are(top left) 1) Empty; 2) 100-mM Yellow-#5 dye; 3) 100-mM Blue-#1 dye; 4)Green dye (blue-yellow combined); 5) DMSO; 6)˜100-mM red-#40 dye. FIG.6B is a bar graph plot of the average pixel value (1,000 pixel average)confirming the lack of any color distinction by NIR.

FIG. 7, left panel, is a scan of a photograph showing NIR images(displayed from upper left to lower right) of precipitate; dissolvedcompound (“normal”); a large air bubble and crystallization in differentwells of NUNC 384 well polypropylene plate. Left panel shows a pixelprofile which transects across each corresponding well of the platedepicting how the signal intensity and shape varies with each phenomena.

FIG. 8, top panel, is a scan of a NIR image showing a NUNC 384polypropylene compound plate imaged by the NIR-HIAPI system. The bottompanel shows row “l” of the plate which contains different artifacts ineach well and these artifacts are correlated to a pixel profiletransect. The empty wells have higher narrow peaks than full wells(height/width variance), while defects such as crystallization show upas a jagged profile. Bubbles typically manifest as an irregular profilewith a sharp signal drop along one or both edges of a well.

FIG. 9 shows 1.4 mL, screw-top tubes imaged using the NIR-HIAPI systemover a 52 mm pathlength, DMSO filled, water filled and empty tubes aredistinct visually and by pixel profile transect.

FIG. 10 is a schematic illustration showing an embodiment of theHigh-resolution Image Acquisition and Processing Instrument (HIAPI)configuration scheme. The white boxes represent subsystems that are usedfor visible light applications on the HIAPI, the boxes represent addedfunctionality for NIR applications and barcode scanning.

FIG. 11 is a schematic illustration showing an embodiment of theNIR-HIAPI configuration scheme. This embodiment shows a configuration ina bottom-read mode. Other embodiments include combination of thesensors, such as, for example combining vis, NIR.

DETAILED DESCRIPTION

High-resolution image acquisition and processing systems for detectingundesirable physical properties that may affect results in assays areprovided. The system comprises one or more of: an image acquisitionsystem; a light diffuser or emitter; sensors; filters; a consumablestaging mechanism; algorithms for the processing, classifying andanalysis of the information; at graphical user interface; interfacingmeans to a computer or computing system.

Embodiments of the systems or instruments detect undesirable physicalproperties (“artifact”) in microtiter plates/tubes in an automatedfashion. Such undesirable physical properties include, but not limitedto: (1) Compound precipitate/crystallization; (2) Full, empty orpartially-filled microtiter plate wells or microtubes; (3) Colored(including fluorescent) compounds; (4) Contaminant, such as water inDMSO; (5) Objects, bubbles, artifacts; (6) Quality of plate itself.Remediation of contaminant may also be desired. These systems, forexample, the High-resolution Image Acquisition and Processing Instrument(HIAPI), accomplishes its analysis by innovative application of machinevision, HTS imaging and, most recently, NIR imaging and contaminantreducing technologies.

In another embodiment, a further system, termed herein, theNear-InfRared-High-resolution Image Acquisition and ProcessingInstrument (NIR-HIAPI), comprises a near infra red camera. This systemhas many advantages, including alleviating the tedious and timeconsuming activities associated with HTS compound library QA/QC. Boththe HIAPI and NIR-HIAPI systems address these issues. Other advantagesinclude, performing measurements in the most common consumables used byHTS and compound management (CM) labs. In one embodiment, the system,for example. HIAPI, comprises a specialized telecentric lens that imagesan entire microtiter plate simultaneously, such that analysis throughputis much faster (<1 minute/plate) than competitive instrumentation thatemploy, for example, nephelometric or acoustic detection technologies.There is no single instrument that is capable of identifying all thephysical properties (artifacts) listed above. Further, the instrument issimple to use, requiring little/no calibration in routine use.

In embodiments of the invention, the HIAPI system performs fivesimultaneous QA measurements in a noninvasive fashion, namely: (a)quantify sample volume; (b) flag empty compound wells, (c) quantifyhydration level of DMSO, (d) detect compound precipitation, and (e)classify potential assay interferants from colorimetric analysis. Inaddition, micro-scale remediation technology, amenable to high density(i.e. 384-wells, 1536-wells) micro-titer plate (MTP) formats, will alsorecover compromised compounds.

Examples of uses for the systems include, but not limited to: (i) rapidand non-destructive analysis of microtiter plates (MTP) and theircompound contents. At image acquisition speeds of 125 Hz, dataacquisition is rapid allowing for simultaneous assessment of all wellsacross a MTP. NIR complements HIAPI by providing alternativeconfirmation of problem wells. The NIR-HIAPI image of the MTP/tubeplastic can reveal potential defects or damage to the plate itself thatmay hamper its proper use with HTS automation, (ii) Quantitativedetermination of DMSO and water quantities: unlike visible lightimaging, DMSO and water are both visible and distinct in the NM range.NIR-HIAPI will quantify water absorbed and provide a spatial record ofhygroscopic changes and edge-effects. (iii) Inventory solution volume:NIR absorbance is linearly dependent on path-length and hence wellvolume. Calibrated micro-titer plates can be quickly assessed forremaining volume of the compound solute. Since low levels of compounds(<100 mM) will not interfere, calibration by plate type is all that isneeded making implementation direct and simple. (iv) QC with respect toprecipitation and homogeneity: detection of compromised wells withtranslucent crystal precipitates can be difficult under standard visiblelight image analysis. However the strong CH-bond multimode vibrationalabsorbance in NIR is easily detected and universal for all organiccompounds. The long wavelength of the NIR region (0.7-2.5 μm) createssignificant light diffraction resulting in a vivid visualization ofdefects such as bubbles, droplets, surface tension changes andcrystallization. All HIAPI instrument with NIR capabilities will analyzemicro-titer plates in less than a minute providing unsurpassedthroughput. HIAPI or NIR-HIAPI analysis time is independent of welldensity (96, 384, 1536 etc) making it forward compatible and adaptable.Unlike HPLC operational costs and materials consumed per assay isessentially nil (electricity). Other uses include, but not limited to:Chemical Imaging, 2D Fourier Transformation, Hyperspectral imaging,multispectral imaging, pixel line analyses, real-time analysis,convolution functional analysis, machine vision and the like.

In a preferred embodiment, the systems can be used at differenttemperatures or have the means to heat, cool, or stabilize thetemperature. For example, a consumable or sample can be heated totemperatures above room temperature, or cooled below room temperatures.In one embodiment, the sample may be placed on a heating or coolingdevice, or the system comprises a chamber that the sample is placed inand the temperature regulated. In other embodiments the sample can besubjected to different types of atmospheres. For example, nitrogen orcarbon dioxide etc., can be pumped into a chamber comprising a sample orconsumable; the atmosphere can be acidic, basic etc. In one embodiment,the chamber is air tight so as to allow for controlled conditions.Oilier examples of temperatures comprise: Ambient, Room temperature, +4°C., −20° C., −80° C., Peltier. Examples of gas baths comprise Ar, N₂ andthe like.

HTS molecular screening requires a considerable financial investment inacquiring and maintaining large and diverse chemical libraries. Incompound management (CM) operations, quality standards are needed toensure the integrity of a HTS chemical library throughout its lifecycle.To date, no universal standard has been adopted that can rapidly accesscompounds in a noninvasive and nondestructive fashion. Pharmaceuticallibraries solvated in DMSO can degrade overtime with water absorptionthrough storage handling processing. Moreover, screening campaigns bearadditional expenses from false results due to screens that includecollections of unacceptable quality, e.g. precipitates, bubbles,volumes, defects in the consumables and the like. The systems describedherein, can bridge this critical need by providing rapid assessment ofquality, quantity and level of hydration.

In a preferred embodiment, an innovative compound library QA/QCinstrument, the High-resolution Image Acquisition and ProcessingInstrument (HIAPI) is shown in FIG. 1. The instrument shown in FIG. 1 ismerely for illustrative purposes and is not meant to be limiting norconstrued as such. Using a hi-resolution CCD camera (100) andtelecentric lens combination (110), it is configured to image the top orbottom of an HTS consumable, e.g. microtiter plate or microtube receiverrack containing compounds. The consumable can be, for example, amicrotiter plate comprising 96-wells, 384 wells, wells or more. Othertypes of consumables can be used without limitation. Once an image iscaptured, the HIAPI employs custom digital-image processing algorithmsto identify the most common artifacts that were observed in the compoundlibraries, such as for example, missed (or partially filled) wells,wells where a compound has precipitated, or wells containing coloredcompounds. This task is performed rapidly, about 1 minute for a 384 wellplate. The system or instrument is also capable of outputting theresults of the analysis to a date file and archiving images to adatabase for future reference.

FIG. 1 is an illustrative example of the HIAPI instrument or system. Inthis example, the HIAPI comprises a color CCD camera (100), imaging lens(110), microplate stage (120) and brightfield illumination system (130).The hardware is mounted on an optical table (140) and connected to ahigh-speed laptop computer via a FireWire interface (150). Some of thedistinguishing features of the HIAPI from industrial vision appliancesinclude the combination of a telecentric lens with a high-resolutionprogressive scan CCD camera. The telecentric lens is capable of imagineall the wells of a standard microtiter plate or tube rack without needfor repositioning the plate and with negligible perspective error; whenthe camera is mounted with the telecentric lens, the WAN is able toresolve features less than 1 mm in diameter with minimal vignetting anda depth of focus that is forgiving to plates that may be slightly out ofANSI-defined tolerances. In the HIAPI configuration embodiment shown inFIG. 1, illumination is provided via coaxial diffuse lighting, which isspecifically suited for imaging applications where the target object ishighly specular in nature. For example, microtiter plates are primarilymanufactured from polypropylene, which fits into this category. Thecamera is communicably linked to a computer or processor. For example,the FireWire based camera was chosen to allow greater independence whenchoosing a software development platform, as the FireWire interface issupported by all major operating systems and imageacquisition/processing/analysis software packages. However, other typesof sensors and interfaces can be used, including wireless interfacing.Examples of sensors (single or array of) comprise; CCD, Photodiode,Photo multiplier tube, IR-NIR arrays, Focal Plane Array, InGaAsphotodetector; VisGaAs photodetector, InSb photodetector, Quantum WellInfrared photodetector and the like.

The sensor, see, for example. FIG. 1 (CCD camera; 100) may be any deviceadapted for magnifying, illuminating and recording a digital image of atleast a portion of the test object containing a target. This device mayhave a built-in magnification and illumination feature or may have anattachment that provides these features. In an embodiment, a lens-baseddevice attachment may be used in conjunction with a standard digitalcamera, CCD etc., to illuminate, magnify and capture a digital image ofan image. In particular, the lens-based device may illuminate andmagnify an image. The lens-based device may include a housing, at leastone light source for illuminating an image in a predetermined frequencyrange, filter(s), and a lens for magnifying the image. Similarlens-based devices, field microscopes or other illuminating and/ormagnifying attachments may be fitted to virtually any form of portableor non-portable digital image capturing, device, including various typesof digital cameras, scanners, cell-phones, PDAs, etc.

In general, the WAN system comprises dual-image analysis using visibleand infrared camera technologies. Custom hardware compatible to futurerobotic integration will apply “machine vision” appliances, with opticsand image analysis software. Near Infrared (NIR) imaging can providequantitative Chemical Imaging (CI) of compound micro-plates and readilydetermine sample volume and hydration levels. Unlike conventional QA/QCtechnologies that perform sequential sample analysis, the HIAPI willperform image analyses simultaneously on the entire MTP field and willnot be time limited to any appreciable extent by the format density(i.e. 96-w, 384-w, 1536-w). Most importantly, compound libraries can besubjected to rapid QA/QC prior to a HTS screening campaign to providecritical and timely validation of the data. In one embodiment,micro-scale remediation technology integrates zeolites with liquidhandler micro-fluidic workstations to provide selective water removaland recovery of precipitated compounds in compromised microtiter storageplates.

Remediation means to remove a contaminant from the sample (e.g. compound& DMSO). In one embodiment, water is a contaminant of a sample and ispreferably removed. For example material (e.g. zeolite, liquidchromatography material) is added to the sample well that will act as a“selective sponge.” Once it is in contact with the sample, it will soakup the water that has contaminated the sample. Alter the water isremoved by the “selective sponge”, the “selective sponge” is removedfrom the sample. Now that the sample has no water in it, compounds thathave precipitated due to water insolubility go back into the DMSOsolution.

The selective sponge can be zeolite, liquid chromatography material orsimilar material; it can be added as a bead suspension directly to thewell, or can be embedded in a pipette tip, capillary, microtiter platelid, etc. It can be attached to an automated mandrel that dips in thewell, for example an automated pipettor, etc.

Although in some embodiments, removal of water is preferred, othercontaminants (e.g. unwanted chemical species such as byproducts of aorganic, synthesis) can also be removed; using different materials toaccomplish this.

In preferred embodiments, the systems described herein, comprise:manual, semi-automated, fully automated systems or combinations thereof.

To date, HTS compatible instrumentation is inadequate or nonexistent toqualify and maintain the vast and ever growing small moleculerepositories used. Chemical Imaging and machine vision analysis isadaptable to future developments in micro-array formatting and easilyintegrated into existing robotic platforms or compound managementsystems. Success in this endeavor will produce technology that can beleveraged to government, academic screening facilities, and thebiotech/pharmaceutical industry to reduce expenses and improve thequality and stewardship of HTS drug discovery libraries.

It should be understood that the programs, processes, methods andsystems described, herein are not related or limited to any particulartype of computer or network system (hardware or software), unlessindicated otherwise. Various types of general purpose or specializedhardware and/or computer systems may be used with or perform operationsin accordance with the teachings described herein.

In a preferred embodiment, light from a light source may be captured bythe sensor, for example, CCD, whether the light is reflected and/ortransmitted. The systems described herein can measure light that hasbeen reflected, emitted, transmitted, refracted, polarized; also,absorbance, transmittance, phosphorescence, luminescence and the like bycompound, artifact contaminant, or the consumable itself. The light canbe any wavelength or can be from a light source emitting, particularwavelengths or frequency ranges. The predetermined frequency rangeincludes ultraviolet light frequency and an infrared light frequency.The light sources may emit a concentrated portion of light on aparticular area of the consumable or sample in a consumable, or may bediffused over, for example a microtiter plate. In a preferredembodiment, the system comprises a device to diffuse light, or mayinclude a function to diffuse light placed in optical alignment with thelight source. The light diffuser device may be any shape. For example,for an even distribution of light over the consumable, the lightdiffuser may be shaped as a “ribbed” cone. Diffuser devices may also bepurchased commercially.

In another preferred embodiment, the wavelength of the light emitted bythe light source may be broadened and/or narrowed by a light filter. Thelight filter may include a colored filter, a split field filter, apolarized filter or any other filter used in digital photography orspectroscopy. The filter can function to assist in viewing and/orcapturing images. The light filter may be a long pass filter, short passfilter, or a band pass filter. A long pass filter functions to transmita wide spectral band of long wavelength radiation thereby blocking shortwavelength radiation. A short pass filter functions to transmit a widespectral band of short wavelength radiation thereby blocking, long,wavelength radiation. Examples of filters comprise without limitation,one or more of: Polarized, Atttenuated, dichroic, emission, excitation,Longpass, Shortpass, Bandpass, Notch Coated, Dielectric, light,diffuser.

The type of light source can be varied, in many cases, the light sourcemay be an LED, incandescent bulb, fluorescent bulb, or halogen bulb.LEDs have advantages because they are typically of small size, but stillproduce a substantial amount of light versus the amount of power theyconsume. The light source may provide constant illumination or amomentary flash timed to coincide with image acquisition. The flashdevice or other light source may include a filter to tailor theillumination spectrum. Power can be delivered, to the light source byany electrical power source, including battery power is preferred tomake the lens-based device mobile and independent of its proximity to astationary power supply, such as an electrical outlet.

In another preferred embodiment, a light source comprises a single lightsource, an array of light sources comprising the same light source or acombination of different light sources. Examples include, withoutlimitation: blackbody radiator, IR, UV, visible, laser, monochromatic,polychromatic, LED, tungsten-halogen, quartz-tungsten halogen. Hg arclamp, Xe arc lamp, or Electroluminescence.

In one embodiment, light is provided by one or more laser sources,and/or an illumination lamp, such as, for example, a xenon (Xe) lamp, ora mercury (Hg) lamp optimized for broadband illumination. Examples oflasers include, but not limited to a solid state laser providing.UV/violet light, such as, for example, a gallium nitride laser diodeoperating at wavelengths in the range of 380 nm to 420 nm; a solid statelaser operating in the red, such as, for example, a diode laseroperating at wavelengths in the range from of 630 nm to 670 nm, such as,for example, an AlGaInP laser diode.

In another embodiment, the light source comprises a laser diodeoperating in the red (e.g., from about 630-670 nm) a laser diodeoperating in the green (e.g. from about 500-550 nm), and a laser diodeoperating in the violet (e.g. from about 380-460 nm). In one version,the red, green and violet laser diodes are operated to provide both“white light” to serve as an illumination light source and excitationlight. For example, corrected broadband illumination (“white light”) canbe provided by controlling the intensity and bandwidth of the light thatreaches the tissue front each the three laser diodes. Excitation lightcould, e.g., then be provided by the violet laser diode, by frequencydoubling the light from the red laser diode (e.g., with a KTiOPO₄(KTP)crystal), or both, to provide one or more excitation wavelengths.

In one embodiment, the system comprises an optical table upon which theconsumable is placed. Although, neither the consumable nor the camera isrequired to move in order to provide the analysis of the consumable orsample in a consumable, the camera may be mounted such as to allowmovement of the camera. In other preferred embodiments, the lightsource, filters, diffusers etc., can be stationary or have the abilityto move in any directions or angles, depending on how the user woulddesire the system to be configured. Examples of different configurationsof the system comprise: stage movable, lens and/or sensor fixed; stagefixed, lens &/or sensor movable, lens fixed, sensor &/or stage movable;epi-illumination imaging, trans-illumination imaging, split-beam dualdetector systems, diffuse axial illumination imaging, directionalillumination imaging, glance illumination imaging, diffuse illuminationimaging, darkfield illumination imaging, backlighting illuminationimaging or any combinations thereof.

The components of the system may be interconnected via any suitablemeans including over a network, e.g. the HIAPI to the processor orcomputing device. The processor may take the form of a portableprocessing device that may be carried by an individual user e.g. laptop, and data can be transmitted to or received from any device, such asfor example, server, laptop, desktop, PDA, cell phone capable ofreceiving data, BLACKBERRY™, and the like. In some embodiments of theinvention, the system and the processor may be integrated into a singleunit. In another example, a wireless device can be used to receive animage and forward it to another processor over a telecommunicationsnetwork, for example, a text or multi-media message.

The functions of the processor need not be carried out on a singleprocessing device. They may, instead be distributed among a plurality ofprocessors, which may be interconnected over a network. Further, theinformation can be encoded using encryption methods, e.g. SSL, prior totransmitting over a network or remote user. The information required fordecoding the captured encoded images taken from test objects may bestored in databases that are accessible to various users over the sameor a different network.

In some embodiments, the data is saved to a data storage device and canbe accessed through a web site. Authorized users can log onto the website, upload scanned images, and immediately receive results on theirbrowser. Results can also be stored in a database for future, reviews.

In some embodiments, a web-based service may be implemented usingstandards for interface and data representation, such as SOAP and XML,to enable third parties to connect their information services andsoftware to the data. This approach would enable seamless datarequest/response flow among diverse platforms and software applications.

In another preferred embodiment, the system comprises a plurality orarray of sensors. Each sensor may differ, for example, a CCD camera anda NIR camera. See, for example, FIG. 11. Other examples of sensorscomprise CCD, Photodiode, Photo multiplier tube, IR-NIR arrays, FocalPlane Array, InGaAs photodetector, VisGaAs photodetector, InSbphotodetector, Quantum Well infrared photodetector or combinationsthereof. In general, the detector system comprises a charge coupleddevice (CCD), a CMOS imaging device or other pixelated imaging sensor.In addition, in preferred embodiments of the systems of the presentinvention, the detector system includes a frame grabber to acquireimages, e.g., of the consumable under investigation. Preferably, imagesare acquired for every reference location in a consumable and consumablecomprising a sample from which spectra are obtained.

In another preferred embodiment, the image may optionally be magnifiedby the image acquisition device or a lens-based device used inconjunction with the image acquisition device. The image acquisitiondevice may include a magnifying lens with magnification capability or anattachment having lens with magnification capability. The magnifyinglens may magnify the image for viewing and/or capturing. The magnifyinglens may allow an image to be viewed and/or captured from about 0.01microns to 10 microns. In some embodiments, the lens may be a 10-60×lens. The lens in lens may be interchangeable and may interact with azoom lens or regular lens of the image acquisition device. The lens mainteract with the flash of an image acquisition device. Further, thelens may interact with the image acquisition device to increase ordecrease the magnification of the image. The magnification of the lensmay be manual or automatic. Additionally, the lens may be a physicallens or an electronic/digital lens. Other examples of lens compriseTelecentric, Achromatic, Wide angle, High F-number, Low F-number, Zoom,Quartz. Borosilicate, glass, plastic, sapphire; imaging fiber-optics orcombinations thereof.

In a preferred embodiment, a magnified digital image of the test objectis captured using the image acquisition device (100). The captureddigital image may include all or a portion of the object. The captureddigital image may be configured so that only the target area (Area ofCreation) is captured or may be configured so that the target area isincluded in a larger view. In either case, the captured image may alsoinclude identifiable orientation marks that allow the identification andproper orientation of the target area portion of the captured digitalimage. The captured digital image may be downloaded to or sent to aprocessor. At the graphical user interface (see, for example, FIG. 2)the captured digital image is viewed and or processed by the processor.Some or all of the processor may be co-located with the inspection site(i.e., the location where the digital image of the test object iscaptured) and some or all of the processor may be remote from theinspection site. In either case, the processor may be connected to theimage acquisition system over a network. The captured digital image maybe transmitted over the network in any manner such as by e-mail or othertransfer process. In some embodiments, the digital image may betransmitted over a wireless telephone or other telecommunicationsnetwork. It can also be sent as an attachment to any form of e-mail ortext or multi-media message.

In another preferred embodiment, the system comprises a user interface.In a preferred embodiment, the user interface is a graphical userinterface. Referring to FIG. 2 which represents one embodiment of thegraphical user interface (GUI), was written in MATLAB®, MATLAB® is ahigh-level technical computing language and interactive environment foralgorithm development, data visualization, data analysis, and numericcomputation. However, any computing programming language can be used indesigning and implementing in the graphical user interface.

A variety of software packages which are commercially available can beemployed in embodiments of the invention. For example, NationalInstruments Vision Builder (NIVB) and MATLAB® for HIAPI's image analysisapplications. Both these software packages provide a library of commonimage processing and analysis toolboxes; these tools allow developmentto focus on implementation of novel solutions by pulling from a libraryof well proven algorithms and functions. Other examples of softwarepackages include: Deblurring, Enhancement, image Arithmetic. ImageRegistration, Image Segmentation, Spatial Transformation, MeasuringImage Features and Transforms processing algorithms. In MATLAB®, thesealgorithms are accessed by calling them as functions with variousparameters, such as an image frame, passed through to the function,HIAPI commands may be issued in a command window for immediateprocessing or written with other commands into MATLAB® M-tiles which canbe later executed. NIVB provides functionality similar to that which isavailable in MATLAB®, however NTVB allows users to construct imagingapplications through a graphical user interface (GUI). This allows theuser to drag and drop components into a workspace, connect them asneeded and set parameters without having to write any code.

In some embodiments, the data processor may implement an analysisprogram and/or functionality of the methods of the present invention assoftware on a general purpose computer. In addition, such a program mayset aside portions of a computer's random access memory to providecontrol logic that affects the analysis program, light source control,detector systems spectra acquisition, and the operations with and on themeasured images. In such an embodiment, the program may be written inany one of a number of high-level languages, such as FORTRAN, PASCAL, C.C++, or BASIC. Further, the program may be written in a script, macro,or functionality embedded in commercially available software, such asEXCEL or VISUAL BASIC. Additionally, the software could be implementedin an assembly language directed to a microprocessor resident on acomputer. For example, the software could be implemented in Intel 80×86assembly language if it were configured to run on an IBM PC or PC clone.The software or code may be embedded on an article of manufactureincluding, but not limited to, computer usable medium such as a harddrive device, a CD-ROM, a DVD-ROM, or a computer diskette, programmablelogic controller, BASIC-STAMP, microchip, etc. having computer readableprogram code segments stored thereon.

It will be apparent to those of ordinary skill in the art that methodsinvolved in the system and method for light scattering spectroscopy maybe embodied in a computer program product that includes a computerusable medium. For example, such a computer usable medium can include areadable memory device, such as, a hard drive device, a CD-ROM, aDVD-ROM, or a computer diskette, having computer readable program codesegments stored thereon. The computer readable medium can also include acommunications or transmission medium, such as, a bus or acommunications link, either optical, wired, or wireless having programcode segments carried thereon as digital or analog data signals.

The processor may be configured to automatically carry out some or allof the remaining steps of the method as described in detail in theexamples section which follows. If necessary, the software or algorithmsmay compare the object or consumable, via the captured image frames orimages, with certain predetermined criteria or controls, which mayinclude an expected control. Also, if the image is an encoded image, theprocessor may decode the image. In such instances, the processor maydetermine one or more of the encoding parameters used to encode theimage. The number of parameters required may depend on the specificdigital decoding methodology used. The encoding parameters may beobtained from data storage where they are placed at the time ofencoding. This data storage may be a part of or co-located with theprocessor or may be disposed in a separate database processor or serveraccessible to the processor over a network. The data storage may alsotake the form of a magnetic stripe, laser card, smart, card, processorchip, memory chip, flash memory, sample ID or bar code, which can beapplied or attached to or otherwise associated with an object to whichart image is applied. The encoding parameters may be object-specific ormay be constant for a particular set of objects. In some embodiments,some or all of the encoding parameters may be received with an encodingrequest or determined from the content of the image.

In some embodiments, the method may be adapted to determine whether thecaptured image or certain captured frames comprise micro-printing orrasters formed as a particular shape. Such printing devices may beidentified in both encoded and non-encoded images.

The processor may use object landmarks to orient the target area of thecaptured digital image for viewing and/or decoding. These landmarks maybe based on the inherent geometry or topology of the object or may bespecifically applied. The presence of such landmarks could be used as aninitial check. It will be understood by those of ordinary skill in theart that if the digital image is captured in such a way that the objectis always oriented in exactly the same way relative to the imageacquisition device, there may be no need for digital orientation of thecaptured image. For example, if the test objects are documents that canbe precisely positioned for scanning, the orientation of the target areamay be sufficiently constant that orientation of the captured digitalimage is unnecessary.

If a comparison is desired, this may involve a sequence of criteriabeginning with whether an image is even present in the target area. Ifan image is present, it may be directly compared to a control image orcompared to any number of images which may have been stored or are beingcaptured as part of the quality control and quality assurance, forexample, variation parameters within a set of consumables ormanufactured lots of consumables or consumables comprising a sample, canbe determined. Thus, if there is a large variation between lots orwithin a lot then these may be eliminated from being sent to a user. Thesamples and consumables can be further processed to provide a resultthat can be compared to a control image or information derivable fromcontrol or images from a manufactured designated lot. Thus, verifyingthe quality of the consumables may comprise, inter alia, the actions ofviewing the captured image an/or comparing it to an expected image,decoding the image, and deriving information from the captured image ora decoded version of the captured image.

Optical magnification may be used in conjunction with the digitaldecoding method to reduce the influence of imperfections in the captureddigital image and improve the ability to sample the captured digitalimage. In some embodiments, the methodology samples one or more lines ofthe captured digital image at certain desired frequencies and angles.For example, one or more sampled lines of the captured digital image maybe combined to generate one line of a result. The optical magnificationof the image can determine the actual pixel spacing between the sampledlines. The physical spacing of the image should match the lines spacingused during the analysis of the control, or the line spacing of theequivalent magnifying lens. The number of pixels between the sampledlines of the magnifying lens is calculated. A physical measurement, suchas picture of a calibration grid, may be used to obtain a scale factorfor the magnifying, lens. The physical measurement may be calculatedautomatically. The methodology enhances the sampled lines of thecaptured digital image to remove any gaps between lines to produce aresult.

Although not necessary with the systems of the invention and embodimentsthereof, when viewing and/or capturing an image one may also considerdetermining the actual pixel-per-inch resolution of the captured image;and compensate for the different types of geometrical distortion thatcan be induced by the image acquisition device. Assuming the imageacquisition device maintains the same distance from the object and thezoom function is not used. For example, the image acquisition device ispositioned directly on the surface of the object thereby providing aconsistent capturing distance. However, if the zoom function is used orthe image acquisition device does not maintain a consistent distance,pre-calculated values are difficult to use. The positions and distancesof the reference points on the object and the scale factors of the imagewill need to be recalculated, for example, when a new type of consumableor sample is to be analyzed.

Numerous methods may be used to determine the actual pixel-per-inchresolution of the captured image. The preferred methods are described indetail in the examples which follow. Two of the methods are usingcalibration to determine the real pixel-to-pixel resolution of the imageand rescaling a frequency.

The digital camera can be calibrated to determine the realpixels-per-inch resolution of the viewed and/or captured image. Thescale factor of the digital camera can be calculated. The scale factorcalculation occurs by taking a picture of a reference pattern, whosephysical dimensions are known. Alternatively or in addition, the imageacquisition device or attached lens device may produce repeatableeffects on captured images that may be used as a reference. For example,a magnifier may limit the captured field to a circle with a known, fixeddiameter. In either case, if there are 1800 pixels covering one inch ofthe reference pattern then the resolution is 1800 pixels-per-inch. Next,the scale factor can be determined by dividing the reference patternresolution by the actual resolution.

Geometrical distortion may also be considered when viewing and/orcapturing an encoded image. Misalignment and/or rotation can distort anobject, however, both can be compensated by decoding software. Thedecoding software can calculate the angle of rotation in the viewedand/or captured image. Of the many methods used to calculated therotation angle one can use the positions of some easily locatedreference points on the object or looking for a maximum of a Radontransform for an image with dominant line structures. Once the rotationangle is calculated, the captured image may be held in its referentposition, to avoid distortion caused by the rotation process (e.g.interpolation on the digital grid blurs the image). The encoded imagedecoding parameters use the adjusted rotation angle. For example, if anencoded image is embedded with 15 degrees screen angle, and it can becalculated that the object in the captured image was rotated by 3degrees the adjusted angle of 15+3=18 degrees should be used for thedecoding algorithm.

After turning on the instrument and launching the HIAPI software, a“start page” GUI interface is displayed (FIG. 2, right panel). Afterverifying that the picture on the screen coincides with how the hardwareis actually set up, the first decision the user makes is whetherinstrument calibration is necessary. Calibration is unnecessary if theHIAPI has already been calibrated with the particular brand ofmicrotiter plate consumable the user wishes to analyze. If calibrationis desired, the user prepares and measures calibration plates (e.g. anempty plate, a plate, filled with DMSO or other appropriate controls) onthe HIAPI and performs a prescribed calibration routine. This normallytakes 10-15 minutes. Once instrument calibration is completed, the userselects the appropriate protocol to run from the GUI, and the imageacquisition/results display window appears (FIG. 2 right panel). Theuser then places the test microtiter plate/tube receiver rack on theimaging stage, selects what artifacts he is interested having the HIAPIclassify, and starts the HIAPI measurement protocol. Since all of thesystems can be fully automated, a consumable can be placed by mechanicalmeans, such as for example, a robotic arm, moving, belt or otherautomated mechanical means, and the appropriate software protocol can beselected via software interface. As the user waits for the HIAPI tofinish its measurement protocol, the following processes are executed bythe HIAPI. FIG. 3 is an illustrative example of a general scheme of suchprocess. In an embodiment of the process, the steps can be broken downinto different steps: (i) image acquisition which involves the use of animaging acquiring device, such as for example, a camera; (ii) imagestandardization which comprises a graphical user interface or softwareautomation; (iii) image analysis and classification which comprisesprocessing an image matrix through algorithms wherein image descriptorsare calculated and used for classifying the information obtained fromthe image; (iv) completion and storage which comprises aggregating,formatting, the data and display of the data by the graphical userinterface or other appropriate means.

(i) HIAPI Image Acquisition.

In one embodiment, immediately after the measurement protocol isinitiated, the HIAPI software captures a frame from the CCD camera andimports this frame into the HIAPI computer's RAM memory. At this pointin the process, the microtiter plate/tube receiver rack image isdigitized as a 3-dimensional matrix. This 3-D matrix can be thought ofas three stacked two-dimensional matrices, where each 2-D matrix (X- andY-axes) represents the 2-D image (length and width) of each color imagedby the CCD (i.e. red, green or blue) and the Z-axis contains the valuesof the three color channels associated with the image. Other colorsdeterminations comprises use of filters like, for example, bandpassfilters, Bayer filters, absorptive filters, dichroic filters, infraredfilters, ultraviolet filters, neutral density filters, lone filters,shortpass filters, guided-mode resonance filters, polarizers, and thelike.

(ii) HIAPI Image Standardization.

After acquisition, HIAPI digital image processing tools subject thecaptured image to a series of filters that standardize its featuresbefore analytical processing begins. This standardization process canconsist of the following steps: (1) Area of Interest (AOI) Creation: Inthis step, sections of microtiter plate (e.g., the border and skin ofthe plate) are not needed for further analyses are discarded. Since thecompound plate/tube receiver rack being interrogated is located in arigid plate stage, the AOI is determined only once (during instrumentcalibration) for each plate type and then fixed within every image thatHIAPI captures thereafter from that particular plate type. Anyinformation outside the AOI is discarded in this step, as it provides nouseful information and increases the amount of time required fordownstream image processing. In the case of the 3D matrix, this isaccomplished by discarding a fixed border in each of the color channels.(2) Application of environmental corrections: Any compensation needed tocorrect for artifacts due to illumination conditions, optical distortionby compound plate features, etc. is corrected in this step. One methodthe HIAPI uses to correct for environmental conditions is to perform“image subtraction.” In this procedure values from the calibration imagematrix (as with the AOI, stored during instrument calibration) aresubtracted from the current image matrix. A simple example of this typeof correction would be if dust or particulate were to settle on thetelecentric lens of the HIAPI. Detected during calibration, it would besubtracted from each image the HIAPI acquired to ensure it was notcategorized as a false-positive. (3) Threshold Determination: This isactually a two step procedure where the RGB image is converted tograyscale and then further into a single, 24) binary image. Theconversion from grayscale to binary compares each pixel value against agiven threshold to determine if the value of that pixel should beultimately represented as a 1 or 0. In some cases, the HIAPI can use ahistogram approach, known as Otsu's method (Otsu IEEE Trans. Sys., Man.,Cyber, 9: 62-66, 1979), to determine threshold. (4) ObjectIdentification and Enhancement: The HIAPI can use built-in libraryfunctions to enhance and identify objects, such as morphologicaloperators. These morphological operators are capable of enhancing andidentifying objects then determining their relevant properties. Theresult of all morphological, operations is an image frame where allobjects have been outlined, filled in and dilated to make final analysispossible on an image where contrast between distinct objects has beenmaximized. In addition to morphological operations, objects are furtheridentified and classified by means of histograms and intensity profiles.The data provided by these histograms and intensity profiles canrepresent unique ‘thumbprints’ which provide information about theshape, color and intensity of items in the wells/tubes of the microtiterplate/microtube rack. (5) Image Segmentation: The last step of imagestandardization the HIAPI can perform is a virtual segmentation of theimage to separate the plate/rack into individual wells/tubes foranalysis. In other words, the 2D binary image matrix is partitioned intoa series of much smaller 2D matrices, with each smaller 2D matrixrepresenting a single well/tube of the plate/rack.

(iii) HIAPI Image Analysis and Classification.

Following image standardization, the data for each 2D binary imagematrix can be processed through algorithms in the HIAPI software whichcalculate image descriptors and use these descriptors to classify theresulting information. This includes (but is not limited to) calculatingboundaries of objects within the image frame and then plotting thedistance versus angle signatures of these boundaries. Signature andboundary information is then analyzed in concert with histograms, imageprofiles and properties extracted through morphological operators inprevious steps to classify the contents of the well. The results of eachwell's classification are recorded into memory and the process isrepeated until all wells have been analyzed.

(IV) HIAPI Performs Image Capture and Analysts Rapidly.

For one microtiter 384-well plate, the time is about 1 minute. Further,the amount of time, required to image and analyze a plate is independentof plate density due to the nature of the system. Specifically, a platecontaining 384 wells has roughly the same number of pixels containingwell information as a 1536 well plate. This is because while the numberof wells has increased by a factor of four, the number of pixelsrepresenting each well has decreased by a factor of four.

(v) Completion of the HIAPI Measurement Protocol.

Once analysis is completed, classification data for the entiremicrotiter plate (or for each well measured) can be aggregated andformatted into a report for the user to view through the HIAPI's GUIinterface. At this point, the user is prompted as to whether to discardthe image or save the image and the corresponding analysis. The user mayalso edit parameters for custom applications. IF the image and resultsare saved, the original source image, along with frames from any of thepreceding steps is stored in the HIAPI's database. A full-color imageand a *.csv file are also saved on the HIAPI computer hard drive, whichis readily accessible to the user if desired.

Image Acquisition, Standardization; Color “Artifact” Determination,Empty-Well “Artifact” Determination.

The HIAPI software processes begin when all image data is imported fromthe CCD camera into HIAPI's computer memory. FIG. 4 illustrates how theHIAPI is capable of enhancing the images of microtiter platewells/microtube receiver rack into meaningful quantitative informationthat enables automated classification. After selection of the AOI, oneof the first steps in the HIAPI performs is the extraction of colorinformation, which is analyzed in terms of pixel intensity at each colorchannel. Clearly visible in the first row of “raw” CCD images in FIG. 4is the presence of a red compound. The resulting histograms from thisimage are immediately used to determine the color(s) present in a well(“red”); the amount of variance in color across a well is alsoimportant, since it combined with other downstream determinations (i.e.“precipitate”) to better classify the type of artifact present in thewell (“red”+“precipitate”=“red precipitate”). Following the extractionof all color channel histogram data, image subtraction is performed as amethod of environmental correction; i.e. HIAPI compares the interrogatedwell to an empty well or deduces that the well is empty based onpre-determined parameters. If a well is empty, it results in a “black”image and there is no further processing necessary. If the well is notempty, then the resulting image highlights the distinguishing featuresquantified in its 2-D matrix.

The next step, illustrated by FIG. 4, is the conversion of the imagesubtraction result into grayscale. This removes color information, whichis necessary for downstream processing. While this is a ‘flattening’ ofthe image from 3-D to 2-D, the intensity values representing objects ofinterest are retained. When the image is at this stage, meaningfulinformation can be obtained in the form of grayscale histograms and“distance” profiles (FIG. 4). Grayscale histograms and distance profilesprovide an assessment of pixel intensity across a selected cross-sectionof the well. In the case of the HIAPI analysis illustrated in FIG. 4,the selected cross-section passes horizontally through the center of thewell. The last stage of image processing uses a threshold to convert thegrayscale image to a binary image. The end result represents the objectspresent in a given well that had an intensity greater than the definedthreshold. These binary objects undergo enhancement routines to removespurious objects which might otherwise be identified as false positivesin downstream analyses. Finally, the enhanced binary objects can be putthrough algorithms to extract and plot their boundaries, known as“signatures” (FIG. 4).

Image Analysis; Precipitate, Partially Filled Well Determination &Classification.

With all of the information from the preceding steps, final imageanalysis combines object signatures, morphological properties,histograms and distance profile information together to determine howthe contents of a well should be classified. From FIG. 4, it can be seenthat certain patterns exist; these and the results of other algorithms)can be distinguished from each other as follows: 1) Partially filledwells have object signatures which are erratic in shape and havesignificant variation in amplitude across the signature. In terms ofmorphological properties, the centroid location will be offset from thecenter of the image and object area will be noticeably less than thearea of the well which is pre-determined during the calibration of theHIAPI. 2) “Normal” wells (those containing enough compound liquid tocover the bottom of the well and without precipitate), have binarizedobjects which look similar to toroids (“donuts”). The signature forthese objects is flat; the centroid of the object analyzed is very closeto the absolute center of the image and its area is equivalent to thearea of the well. Calculation of an Euler number also aids in theanalysis of these images; in this case it represents the number ofobjects detected in the image, minus the number of holes in the objects.“Normal” wells have an Euler number of about 0.3. Empty wells will berepresented by a black image frame after image subtraction in all cases,so no further properties are necessary for classification. Wellscontaining, precipitate are represented by a solid circular object inthe binarized image and have flat signatures, similar to non-problematicwells. However the amplitude of the signatures is significantly lowerthan seen in the signature of a non-problematic well. These wellstypically also have an Euler number of 1, an object area equivalent tohalf the area of the well and a centroid located in the center of thewell.

The HIAPI is fully capable of automated artifact analysis in a varietyof HTS consumables. See, the Examples section which follows and Table 1.

In a preferred embodiment, one or more filters are employed in the HIAPIand NIR-HIAPI systems. For example, use of optical filters (e.g., coldfilters) that block the visible wavelengths (0.4-0.78 μm),charge-coupled devices (CCDs) used in digital cameras and camcorders canbe used to sense NIR light out to around 1100 nm. Other regions of theNIR spectrum can be viewed using devices such as indium gallium arsenide(InGaAs—0.9 μm to 1.7 μm) and indium antimonide (InSb—1.0 μm to 5.0 μm)focal plane array (FPA) detectors. These integrated wavelength NIRimaging approaches allow one to study relative light intensities ofobjects over broad ranges of the NIR spectrum.

In another preferred embodiment, the filters comprise one or more of:Polarized, Attenuated, dichroic, emission, excitation, Longpass,Shortpass, Band pass, Notch Coated, Dielectric, light diffuser,polarizing or combinations thereof.

The use of dielectric interference filters in combination with NIR FPAsis one method in which chemical information can be obtained from asample. To form NIR chemical images, a NIR light beam is defocused toilluminate a wide field of view and the reflected or transmitted lightfrom the illuminated area is imaged onto a two-dimensional NIR detector.A selection of discrete dielectric interference filters provided in afilter wheel, or a linearly variable or circularly variable format canbe positioned in front of a broadband NIR light source, or in front ofthe NIR FPA itself in order to collect NIR wavelength resolved images.Typically, the use of several fixed bandpass filters is sometimes neededto access the entire NIR spectrum. The spatial resolution of the NIRimage approaches that of the optical microscope, while spectralresolution of several nanometers has been demonstrated.

Acousto-optic tunable filters (AOTFs) have been employed asno-moving-parts imaging spectrometers for NIR imaging. The AOTF is asolid-state device that is capable of functioning from the UV to themid-IR depending on the choice of the filter's crystal material.Operation of the AOTF is based on the interaction of light with atraveling acoustic sound wave in an anisotropic crystal medium. Theincident light is diffracted with a narrow spectral bandpass when an RFsignal is applied to the device. By changing the applied RF frequencyunder computer control the spectral passband can be tuned rapidly withthe benefit of non-moving parts.

In another preferred embodiment, imaging can combine diffraction-limitedspatial resolution with high spectral resolution allowing for collectionof high resolution (spectral and spatial) data using liquid crystal (LC)imaging spectrometers. In general, LC devices providediffraction-limited spatial resolution. The spectral resolution of theLC imaging spectrometer is comparable to that provided by dispersivemonochromator and Fourier transform interferometers. In addition, LCtechnology provides high out of band rejection, broad free spectralrange, moderate transmittance, high overall etendue and highlyreproducible random access computer controlled tuning.

Under normal NIR imaging operation, LC imaging spectrometers allow NIRchemical images of samples to be recorded at discrete wavelengths(energies). A spectrum is generated corresponding, to thousands ofspatial locations at the sample surface by tuning the LC imagingspectrometer over a range of wavelengths and collecting NIR imagessystematically. Contrast is generated in the images based on therelative amounts of NIR absorption, transmittance or reflectance that isgenerated by the different species located throughout the sample. Sincea high quality NIR spectrum is generated for each pixel location, a widevariety of chernometric analysis tools, both univariate andmultivariate, can be applied to the NIR image data to extract pertinentinformation. Correlative multivariate routines are particularly powerfulwhen applied to chemical images collected from samples intentionallyseeded with a known standard material. This approach of incorporatingcalibration standards within an image field of view can be extended toquantitative chemical image analysis. In addition, digital imageanalysis procedures can also be applied to high image quality NIRchemical images to perform routine particle analysis in both two (2D)and three (3D) spatial dimensions. Volumetric 3D NIR chemical imageanalysis can be performed very effectively using numerical deconvolutioncomputational strategies.

Without further elaboration, it is believed that one skilled in the artcan, using the preceding description, utilize the present invention toits fullest extent. The following examples are offered by way ofillustration, not by way of limitation. While specific examples havebeen provided, the above description is illustrative and notrestrictive. Any one or more of the features of the previously describedembodiments can be combined in any manner with one or more features ofany other embodiments in the present invention. Furthermore, manyvariations of the invention will become apparent to those skilled in theart upon review of the specification.

All publications and patent documents cited in this application areincorporated by reference in pertinent part for all purposes to the sameextent as if each individual publication or patent document were soindividually denoted. By their citation of various references in thisdocument. Applicants do not admit any particular reference is “priorart” to their invention.

EXAMPLES

The following non-limiting Examples serve to illustrate selectedembodiments of the invention. It will be appreciated that variations inproportions and alternatives in elements of the components shown will beapparent to those skilled in the art and are within the scope ofembodiments of the present invention.

NIR-HIAPI Instrument Fabrication:

NIR imaging in a transmission-mode requires that the near-infrared lightsource be placed to penetrate through a MTP for measurement by a NIRdetector on the opposite side. Transmission mode measurements provide anaverage signal from the depth (z-axis) while maintaining spatialseparation across the pixel plane (x, y axis). Due to the nature of NIRand its weak absorbance interactions, transmittance mode measurementsare well suited to ascertain solvent (e.g. DMSO) and contaminant (e.g.water) content. Several key pieces of hardware can be incorporated intoa HIAPI platform (FIG. 10).

Camera Integration:

State of the art NIR cameras use InGaAs focal plane arrays (FPA) withoutstanding sensitivity in the 900-1700-μm range. Unlike earlier NIRcameras, no special cryogenic support hardware was needed, simplifyingintegration and support. Data acquired using a 320×256 FPA providedadequate resolution for prototyping and testing. High performance NIRcameras are currently available using 640×512 FPA which increase pixelresolution by 4-fold, desirable for full plate imaging. Newer NIRcameras are capable of a programmable frame rate of 1-125 Hz, 14 bitdynamic range and a 50 MHz data transfer rate allowing for rapid imageacquisition and minimal MTP exposure to the environment and infraredheat. Cameras with combined visible light and NIR sensitivity are alsoavailable. The FPA sensors are of the ⅔″ or 1″ array types requiring alamer lens mount (45 min or F-mount) than the smaller CCD VIS detectors(C-mount). See, for example. FIG. 11 for optimal placement of the twocameras in a prototype.

NIR Camera/Telecentric Lens:

NIR light in the 0.7-1.7 micron range is highly compatible with moststandard glass optical components used for visible light imaging.Optical lens/filters made of plastics should be avoided to minimize NIRloss. Telecentric lenses also require customized optical coatings tocorrect for chromatic aberration and focal plane when using NIR light.However said, the corrections needed produced an acceptable VIS imagewhen properly coupled to a higher resolution (5 to 8-MP) CCD usedcamera. A customized NIR telecentric lens was obtained to evaluate NIRand VIS camera performance. A single optical train with image splittercan also be pursued. In another alternative, the HIAPI can have twoseparate optical trains for dual camera integration into a singleinstrumental platform. Another option is to select a NM camera with anextended VIS scan range of 0.4-1.7 microns, with RGB filters on thefilter wheel to extract colorimetric in formation.

QTH NIR Light Source/Diffuser:

NIR as well as VIS light is efficiently generated by most blackbodysources including quartz tungsten halogen lamps. DC-regulated fiberoptic illuminators with low output power ripple (<0.4%) are readilyavailable for machine vision lighting. Variable intensity control andfilter mounts may also required to select VIS and NIR bandwidths whileeliminating infrared heat. The HIAPI also incorporated two opticaltrains, such that the light source was inserted (see FIG. 1) between theMTP and telecentric lens of the VIS CCD camera end for reflectance, andprovided NIR light for transmission mode Chemical imaging on theopposite side. However, a single optical train for both cameras mayrequire two light sources (reflectance; transmission mode) to providefor sequential image acquisition.

Filter Sets:

Greater quantitative accuracy for DMSO and water measurements usedoptical filters to produce two images: principally in the CH vibrationalmodes; and in the OH vibrational modes. This dual image analysis avoidedcovariance absorbance issues by separating the DMSO signal from that ofwater. Commercially available NIR interference filters are available innarrow and broad band-pass arrangements. These filters were implementedusing a filter wheel either directly from the light source or within thecamera body. Camera integration of the filters provided better NIRimaging but can introduce optical aberrations (image sharpness) andfocal changes due to refractive index modification. Visible light filtersets can also be used to measure absorbance of light at definedbandpasses to aid in artifact detection or color determination.

Data Analysis and Software Tool Development:

it has been shown that a quantitative relationship for DMSO, water andvolume content can be derived by simply using an unaided NIR cameraimage. However covariance (Δ water with Δ volume) was not addressed.This issue was resolved by separating the DMSO hand from the water bandusing a filter wheel. To resolve two overlapping spectra contributions atwo point absorbance measurement was needed. NIR band-pass filters forsignals at the 1.18μ and 1.55μ absorbance bands resolved DMSO and waterrespectively. Since two unknowns and two measurements are made, aBeer-Lambert derivation was applied to calculate the finalconcentrations [A=absorbance, e=molar absorptivity constant,b=pathlength and c=concentration].A _(1.18μ) =A _(water) +A _(DMSO) =e _(water) *b*C _(water) +e _(DMSO)*b*C _(DMSO)A _(1.55μ) =A _(water) +A _(DMSO) =e _(water) *b*C _(water) +e _(DMSO)*b*C _(DMSO)

Given the additional factor of pathlength, Simplex Optimization for anunderdetermined system can be applied. Consequently two chemical imagesfollowed by image processing allow for well-volume and DMSO-hydrationdetermination to be derived. The HIAPI system was validated from anumber of MTP types (table 1). Since MTP well profiles vary vendor tovendor), non-linear relationships between well volume and depth wasexpected. Absorbance measurements with respect to quantitative DMSO andwater analysis of nonlinear regressions dedicated for each plate typeand manufacturer was conducted. For this work, a calibration set of DMSOsamples with water varying from 0% to 30% was used for regression modeltraining. Evaluation was performed using the same samples doped andundoped with 100 mM compound mixture (TBD) and placed randomly withinvarious test plates. The analysis was deemed successful if reportedquantities were within 10% of the actual value and reproducibility ofthe measurement was within ˜3% variance. Finally, actual compound plateswill be evaluated to generate a plate map view of DMSO hydration.Results were compared to destructive sampling of select wells usingrefractive index measurements or an acoustic auditing appliance duringthe beta-testing phase.

When a NIR camera EPA was aligned with MTP well rows, signal transectsacross select pixel rows provided a wealth of information as discussedabove. Normal wells with dissolved compound produced strong even plateauthat illuminated the entire well, producing a wide-flat peak.Crystallization produced jagged peaks while air bubbles manifested asirregular shaped plateau, often clipped and narrow in width. Firstderivative analysis of pixel transected across well rows using minimalthresholds setting for height and width was developed for rapid andsimplified QC determination of the compound well state. Large periodicinflection points mark solution/wall boundaries and were easilyidentified by first derivative calculations that triggered secondaryinspection within the well domain for additional points of inflections.Given set threshold values, these secondary points of inflectionsignaled formatting problems such as crystallization or dropletformation.

Hyperspectral imaging is an advanced application development and part ofa class of techniques commonly referred to as spectral imaging. With theapplication of heat-map image acquisition of optically filtered light aset of ‘spectral images’ was collected. Each image represented a rangein the NIR spectral band. These ‘images’ were then combined and formed athree dimensional hyperspectral cube for processing and analysis.Hyperspectral documentation to generate plate libraries provided a meansof comparing plate changes through time. The primary advantages tohyperspectral imaging is that multiple spectra is acquired at each pixelpoint, eliminating the need for a priori knowledge of the sample andallowing for post-processing damsel to be mining. Differentialhyperspectral image analysis was used as QC test for compound platefidelity.

Instrument Calibration and Evaluation:

The software process of analyzing a microtiter plate for artifacts wasbroken up into three major sections: image acquisition, imagestandardization and image analysis. After image acquisition andpreprocessing steps were completed, the HIAPI analysis routine took theprocessed frames and inspected individual wells. The goal of thisinspection was to classify each well by using characteristics availablefrom the preprocessed image frame to determine compound/well problems(bubbles, ppt., empty, etc.). An initial prototyping of theseclassifications involved determining a set of static thresholds for agiven microtiter plate. The calibrations may require validation from anumber of MTP types (Table I).

MTP Vendor/Type Volume Plate Defects & Artifact % Water Color VendorFormat Material Full* Partial** Empty* Plate Bubbles ppt. DMSO* RBGGREINER 384-w PP NIR/VIS NIR NIR/VIS NIR/VIS NIR/VIS NIR/VIS NIR VISGREINER 1536-w  PP NIR/VIS NIR NIR/VIS NIR/VIS NIR/VIS NIR/VIS NIR VISCOSTAR 384-w PP NIR/VIS NIR NIR/VIS NIR/VIS NIR/VIS NIR/VIS NIR VISCOSTAR  96-w PP NIR/VIS NIR NIR/VIS NIR/VIS NIR/VIS NIR/VIS NIR VIS NUNC384-w PP NIR/VIS NIR NIR/VIS NIR/VIS NIR/VIS NIR/VIS NIR VIS REMP 384-wPP NIR/VIS NIR NIR/VIS NIR/VIS NIR/VIS NIR/VIS NIR VIS AURORA 384-w COCNIR/VIS NIR NIR/VIS NIR/VIS NIR/VIS NIR/VIS NIR VIS *Only determined aspresent or missing, **NIR mode provides exact quantitative values.

Success was measured by comparing software detection and tagging resultsof defect wells against the known pattern generated by a robotic liquidworkstation or another appropriate method. Calibration and evaluationfor sample precipitation can use a hydrophilic compound that can beforced out of solution by the addition of DMSO. For example, bicucullinemethobromide was one such compound among many that had been identifiedto be problematic, in remaining soluble in DMSO. This method wasvalidated by HPLC-MS to measure “compound as solute” prior and postprecipitation. An alternative methodology can be borrowed from anaccepted nephelometry protocol where an aliquot of silica suspension isdispensed with a liquid-handling workstation into a DMSO well to producestrong light scattering as a colloid.

Development of a Robust, User-Friendly Database Query Tool:

The HIAPI results database is important for documentation and review ofHIAPI results within the user-interface software. The database itselffunctioned as a repository of experimental parameters and results forarchival and documentation purposes. It was also useful for ad hocqueries to identify wells with particular features of interest. Forexample, queries to identify all “blue” wells returned a list ofcompound IDs, along with associated images and other features the usermay be interested in. Queries were also conducted using SQL commands ina non-user-friendly interface; however, other embodiments address theconstruction of a user-friendly query tool that facilitated customqueries of the database. Creation of report templates by the HIAPI, userpopulated with selected HIAPI results, was also performed.

CCD Image Analysis; Precipitate and Artifacts:

CCD images of plates (MTP) were taken using visible light, and wellanalysis was executed for various defects and artifacts, as shown inFIG. 4. Presented are five well types including a full DMSO controlwell, partial filled well with bubble, colored compound, precipitatedcompound, and an empty well. Machine vision processing is emulated usingMATLAB software and image analysis applications to convert the images tosignals that can be translated by computers in real-time.

A six step process was used to reduce the initial image into an “objectsignature” (see last row of FIG. 4) with a digital threshold that isclearly distinct for each well type. In the first step, imagesubtraction front a completely empty MTP was performed. Normal wells(DMSO) and normal wells with colored compounds have binarized imageobjects which looked similar to toroids (i.e. donuts). The “objectsignature” for these was a flat plateau; the centroid of the objectanalyzed was very close to the absolute center of the image and its areais equivalent to the area of the well.

Calculation of an Euler number also aided in the analysis of theseimages; in this case it represented the number of objects detected inthe image, minus the number of holes in the objects. Partially filledwells have “object signatures” which are erratic in shape and havesignificant variation in amplitude across the field. In terms ofmorphological properties, the centroid location can be offset from thecenter of the image and the object area can be noticeably less than thearea of the well. Wells with precipitate are represented by a solidcircular object in the binarized image and have flat signatures, similarto non-problematic wells. However the amplitude of the signatures wassignificantly lower than seen in the signature of a non-problematicwell. These wells also had a different Euler number, an object areaequivalent to half the area of the well and a centroid located in thecenter. Empty wells had a black image frame abler image subtraction froma control plate, so no further properties were necessary forclassification. The “object signature” was zero across the field andquite different from the normal DMSO filled wells.

Image Processing:

Image processing began when all image data was imported to analyze MTPwells as selected “Areas of Interest” (AOI). As a first step theextraction of color information was performed, which is evaluated interms of pixel intensity at each color channel. Clearly visible in thefirst row of “raw” CCD images (FIG. 4) was the presence of a redcompound. The resulting, color channel histograms from this image wereimmediately used to determine the color(s) present in a well and anyvariance across, since it will combine later with other downstreamdeterminations to better classify the well type present (e.g.“red”+“precipitate”=“red precipitate”). Following the extraction of allcolor channel data, image subtraction was performed from an emptycontrol plate. If a test well was empty, it resulted in a “black” imageand there was no further processing necessary. If the well was notempty, then the resulting image highlighted the distinguishing featuresquantified in its 2-D matrix. Note that extraneous information, such asthe data representing plastic around the well was not present in thisimage. The next step illustrated was the conversion of the subtractedimage into grayscale (FIG. 4). This removed color information, which wasnot necessary for downstream processing. While this rendered the datamatrices representing the image from 3-D to 2-D, the intensity valuesrepresenting objects of interest were retained. When the image is atthis stage, meaningful information can be obtained in the form ofgrayscale histograms and “distance” profiles. Grayscale histograms anddistance profiles provided an assessment of pixel intensity across aselected cross-section of the well. In the case of MTP analysis, theselected cross-section passed horizontally through the center of thewell. The last stage of image processing used a threshold to convert thegrayscale image to a binary image. The end result represented theobjects present in a given well that had an intensity greater than thedefined threshold. These binary objects underwent enhancement routinesto remove spurious objects which might otherwise be identified as falsepositives in downstream analyses. Finally, the enhanced binary objectswas put through algorithms to extract and plot their boundaries, knownas “object signatures”. These results were successful in identifying thestate of a MTP compound well and its artifacts. When optimized, themethodology lends itself quickly to frill automation.

Near-Infrared (NIR) Analysis:

The NIR range at ˜0.7-2.5 microns is at an intermediate region of theelectromagnetic spectrum, between the imaging, capabilities of visiblelight and the chemical fingerprinting of infrared spectroscopy. The datapresented both spectroscopic and imaging features that can be exploitedfor nondestructive analysis of HTS compound collections. Weak absorptionarises from overtone and combination stretching/bending of the O—H andC—H bonds. These weak NIR interactions are an important asset, allowingfor deep penetration into a sample for both qualitative and quantitativeinformation. NIR analysis was well suited for water content analysis ofdimethyl sulfoxide (DMSO), without interference from the solvatedpharmaceutical compound. To demonstrate these features, DMSO and watersamples were analyzed in the NIR region as presented. A calibration setof eight samples was prepared to span the 0-30% range for water in DMSO(v/v). Calibration samples were placed into 1-cm spectrometric cuvetteand analyzed on an Ocean Optics NIRD121 spectrometer and data in FIG. 5Gpresents the raw NIR absorbance scans.

Water vibrational bands at 1.45-1.55-μm were assigned to the firstcombination and the first overtone vibrational modes of the OH-bond.Although some overlap with CH-vibrational bands can occur, the 1.55-μmregion was relatively free of any interference demonstrating excellentlinearity with water content (FIG. 5H).

DMSO Analysis was also possible from several CH absorption bands,including the 1.69 and 1.71-μm doublet and the 1.18-μm singlet that canbe assigned to first combination and first overtone vibrational modes.In particular, the 1.18-μm band was unique to CH vibrational, modes andrelatively free of water interference as shown in FIG. 5A. A linearcalibration can therefore be performed for DMSO with a diminishingseries of absorbance peaks at 1.18-μm as shown in FIG. 5B.

MTP and NIR absorbance occurred as the plastic (typically polypropylene)contributed to the 1.69 and 1.71 μm, 1.18 μm singlet vibrational modesfor CH stretch as shown in FIG. 5I. This absorbance contribution was afixed constant that was corrected, by signal subtraction from a controlplate from each plate type.

Compound Interference is unlikely since their concentrations are no morethan 100 milli-molar and are predicted to produce a CH vibrationalabsorbance >2 decades below that of DMSO. Consequently, quantifyingwater and DMSO in the NIR region was free of interference from solvatedcompounds. This is demonstrated in FIG. 5J where a sample of 100-mMacetaminophen in 75% DMSO (water balance) was compared against a DMSOcontrol.

In summary, NIR spectroscopy was well suited for the QC analysis ofcompounds stored in plastic, consumables.

NIR Chemical Imaging (CI) Platform:

Recent developments in NIR imaging technology have led to theadvancement of InGaAs focal plane arrays (EPA) with outstandingsensitivity in the 900-1700-μm range. A SC4000 SWIR FLIR camera with a320×256 pixel array was selected for this work and has a high speedreadout (420 Hz) without the need for any special cryogenic support. Thecamera was coupled to a 0.08× primary magnification telocentric lens. Aquartz tungsten-halogen (QTH) lamp with a light diffuser was used as ablackbody NIR radiator. Optical analysis was performed in transmissionmode.

Wafer and DMSO by NIR Chemical Imaging: NIR-imaging can be used forqualitative assessment with DMSO volume was tested using a 384-wpolypropylene plate (Costar 3965) and filled with varying levels ofrespect to sample volume and levels of hydration. To demonstrate thiscapability, a 384 well MTP plate was selected and filled with varyinglevels DMSO and water. Image analysis of the central AOI (6,400 pixels)of each well provided the average profile intensity for each volume.

DMSO volume was tested using a 384-w polypropylene plate (Costar 3965)and filled with varying levels of pure DMSO. The absorbance gradientobserved from the top left well to the bottom right is principally dueto CH overtone/combinational vibrational mode absorbance. Note that theimages produce a cross-like pattern (FIG. 5E) due to the additional CHovertone/combinational contributions from the polypropylene plate. DMSOspatial analysis of the central 6400 pixels for each well provides anaverage volumetric profile intensity that has a linear relationship todepth.

The same analysis was applied for qualitative assessment of compoundplates with respect to water content. As shown in FIG. 5F, the same384-w polypropylene plate (Costar 3965) was selected and filled to a setvolume of 50-μL for all wells tested. Several mixtures of water to DMSO(Top left to bottom right: 0%, 5%, 10%, 15%, 20%, 25%) were tested.Using the protocol described above, the profile intensity of each wellwas determined and the data was fined to a second order curve (FIG. 5F).Unlike the NIR spectrophotometer, a linear relationship was not founddue to the additional contribution of CH vibrational modes from theDMSO. Future work can use optical filters on the NIR camera to separatebandwidths for OH and CH vibrational modes. Nevertheless, the unalteredNIR camera tested demonstrates “proof-of-concept” with respect toChemical Imaging and quantitative analysis that can be extrapolated fromNIR images.

Compound Interferences and NIR Chemical Imaging:

Solvated pharmaceuticals in DMSO are generally less than 100 millimolarand were predicted to produce a CH vibrational absorbance >2 decadesbelow that of DMSO. FIG. 6A presents two 6-well images juxtaposed topresent a “visible light” image and a “NIR” image of the identical MTPwells. From upper left to lower right corners the test wells were filledusing color dyes as described (FIG. 6A). Although the images werevividly distinct in the visible light CCD imaging, they wereunremarkable by NIR imaging. A bar graph plot (FIG. 6B) of the averagepixel value (1,000 pixel average) also confirmed the lack of any colordistinction. The lack of significant absorbance also applies tocolorless compounds.

The HIAPI enhances the visibility compound precipitate, crystals andbubbles. FIG. 7 is a NIR image of four wells with these differentartifacts present. Noteworthy is that the dissolved compound produced noabsorbance; however concentrated compounds (i.e. crystals) exhibited astrong NIR absorbance (CH combinational/overtone modes). The pixelsignal transect of the four imaged wells clearly indicated profilesunique to each artifact deviations. The “normal well” with dissolvedcompound produced a strong, even plateau that illuminated the full well,corresponding to a wide-flat peak. Crystallization produced severaljagged peaks while air bubbles manifested as irregular shaped plateau,clipped and narrow in width. First derivative analysis of pixeltransects across well rows with minimal thresholds for height and widthwas developed for rapid QC determination of these different compoundstates.

Despite its lower resolution, the NIR camera's 320×256 pixel array canbe used for full-plate analysis. Shown in FIG. 8 is the NIR image from aNUNC 384 compound plate (Cat#264573) with several artifacts. When imagedin the NIR-HIAPI (top-view imaging), various artifacts are clearlyvisible. Simple pixel transect profiling distinguishes wells containingartifact: specifically, the pixel transect of row “I” and demonstrates asimple “visual to digital” conversion of information.

The ability of NIR to penetrate deeply into a sample can be used toanalyze microtubes, another type of popular consumable used for storageof HTS compound libraries (FIG. 9). For this experiment, three tubeswere assembled and sealed prior to imaging. As shown in FIG. 9, fromright to left these tubes are 100% water; 100% DMSO and empty.

NIR imaging was performed across a variety of tube and plate types asmanufactured by different vendors. NIR images were performed in NIRtransmission mode in “top-down” and “bottom-up” viewpoints. Four rows ofsamples were prepared to evaluate HIAPI capabilities with respect tocompound interference; well volume; precipitation detection and solventcomposition (Water/DMSO). The results are detailed in Table 1, above.

MTP Plate Automation HIAPI Instrument:

The HIAPI can be automated, for example, with respect to plate handling.A number of commercial products are amenable to a plate storage andretrieval system, with hardware to pick plates from the storage device(e.g. a plate carousel) and place them on the HIAPI stage for analysis,then return the plate to the storage device after analysis has finished.

Advance Calorimetric Analysis Tools:

Color determination can aid HTS assay screening better when translatedto wavelength transmission for spectroscopic interference determination.Two post-image analysis procedures comprise an algorithm that takes raw,unprocessed data from the camera's pixel sensors and processes it withknowledge of the Bayer filter spectral profile. The Bayer color filterarray is a popular format for digital VIS CCD based acquisition of colorimages. Correlating RGB A/D signal values relative to the spectralfunctions would provide a direct way of estimating wavelength fromcolor. An alternative approach would be to take a processed frame andextract color information by examining color space (RGB, HIS, CIE) data.A user could define boundaries representing colors or wavelengths ofinterest as These boundaries would then be used to partition colorassignments into wavelength domains.

Advance Image Analysis Tools:

In addition to specific techniques discussed, the overall usefulness andusability of the instrument was increased by applying machine learningtechniques. The QA measurements were based on pre-determined thresholdsset for each MTP type and tuned with calibration standards. However,advanced versions of HIAPI analyses can allow the user to interactivelyclassify individual wells within a microtiter plate through a GUIinterface. The data gathered during this pre-inspection classificationcan then be processed using machine learning algorithms to allow quicktraining of unique artifacts, generating better accuracy and allowingfor the addition of alternative labware. These same machine learningalgorithms can provide a post-inspection QC option, which provides anoperator a means to verify QA measurements and make any necessarycorrections within the same interactive GUI. Corrections would then beadded to a training knowledge base and used to increase the overallaccuracy of the instrument. By extending the classification to includemachine learning algorithms, the HIAPI platform can also provide supportfor novel uses (e.g. micro-arrays, tubes) and detect previouslyundefined artifacts. An example of a new/undefined artifact may includeanalysis of MTP mold deformities (e.g. imbedded defects, stressfractures, broken plates), or extended use to HTS bio-assay plates forQC validation.

MTP Remediation Technologies:

Micro-scale remediation will examine the integration of drying agentswith liquid handler workstations to provide selective water removal andrecovery of precipitated compounds in MTP plates. Current methods usingultra-sonication to re-dissolve precipitated compounds generally fail,since increased. DMSO hydration is often the primary cause of deceasesolubility. Only the removal of water can remediate the problem. TheTable below provides an illustrative example of potential drying agents.

TABLE 3 g H₂O/g Reaction Drying Agents Suitable for Drying DesiccantRegeneration Mechanism Aluminum Oxide Hydrocarbons, air, ammonia, arogn,helium 0.2 175° C. Chemisorption nitrogen, oxygen, Freon, H₂O, CO₂, SO₂Adsorption Cupric Sulfate Esters, alcohols, benzene and toluene 0.6 200°C. Hydration DRIERITE, Regular Air, gases, organic liquids and solids.0.066 210° C. Hydration Molecular Sieve Type 3A Molecules of diameter >3 Å: Considered a 0.18 117-260° C.    Adsorption K12[(AlO2)12(SiO2)12]•XH2O general-purpose drying agent in polar and nonpolar media MolecularSieve Type 4A Molecules of diameter > 4Å: Generally 0.18 250° C.Adsorption Na12[(AlO2)12(SiO2)12•X considered a universal drying agentin polar H2O and nonpolar media Molecular Sieve Type 13X Molecules of >10A: used for drying 0.25 200-315° C.    AdsorptionNa86[(AlO2)86(SiO2)106]•X hydrocarbons H2O Silca Gel Na2SiO3•xH2O Mostorganics 0.2 200-350° C.    Adsorption Zinc Chloride Hydrocarbons 0.2110° C. Hydration

In particular, Zeolites are molecular sieves (3A & 4A) that arehighly-selective for water and have been used to dry DMSO to 10-ppm46 orless. It is estimated that only ˜15-mg of a 4A sieve is needed to dry a10-μL sample of ˜30% water in DMSO from a 1536 compound plate. Zeolitesare available as powders (˜5-10 micron) or as extruded, pellets/spheres.Spheres of 14-30 mesh site are commercially available with a 0.6-1.4 mmdiameter. Such spheres can be easily packed into disposablemicro-pipette tips, without significant flow interference, and used witha liquid handler workstation to automate the liquid withdrawn. Dryingagents can also be obtained with a visible indicator for moisture usingammonium tetrachlorocobaltate(II) (NH4)₂CoCl₄ or cobalt chloride CoCl₂.This will cause the agent to be blue when dry and pink when hydrated forvisual verification. Several drying agents (table 3) are selected fortesting on disposable tips for 384-w and 1536-w plates. Water removalfrom DMSO can be tested using either NIR analysis or through refractiveindex measurements, leading to direct testing of precipitated compoundsin MTP wells. To drive solubility to completeness, each treated platecan be resealed and heated using a plate sonicator. Plates can bevisually inspected after 24-hrs post treatment for precipitation and canbe used to QA the molecular mass and quantity.

Although the invention has been illustrated and described with respectto one or more implementations, equivalent alterations and modificationswill occur to others skilled in the art upon the reading andunderstanding of this specification and the annexed drawings. Inaddition, while a particular feature of the invention may have beendisclosed with respect to only one of several implementations, suchfeature may be combined with one or more other features of the otherimplementations as may be desired and advantageous for any given orparticular application.

The Abstract of the Disclosure is provided to allow the reader toquickly ascertain the nature of the technical disclosure. It issubmitted with the understanding that it will not be used to interpretor limit the scope or meaning of the following claims.

What is claimed:
 1. A system comprising: a full-spectrum, ultraviolet,visible, infrared or near infra red light source for irradiating aconsumable or consumable comprising a sample; at least one lens inoptical alignment with light reflected, transmitted through or emittedfrom the sample, wherein the at least one lens comprises a telecentriclens in combination with a high-resolution progressive scancharge-coupled device (CCD); at least one sensor for detecting reflectedlight from the sample or light transmitted through the sample; acomputer or processor running software for processing and analyzing thereflected or transmitted light.
 2. The system of claim 1, furthercomprising at least one filter in optical alignment with the lightsource or sensor for receiving light emitted from the light source. 3.The system of claim 2, wherein a filter comprising at least one of:AOTF, liquid crystal, polarized, attenuated, dichroic, emission,excitation, longpass, shortpass, bandpass, notch coated, dielectric,Bayer filter, or light diffuser.
 4. The system of claim 1, wherein thelight source, comprising: blackbody radiator, IR, UV, visible, laser,monochromatic, polychromatic, LED, tungsten-halogen, quartz-tungstenhalogen, Hg arc lamp, Xe arc lamp, electroluminescence or combinationsthereof.
 5. The system of claim 1, wherein the at least one sensor is inoptical alignment with at least one lens for focusing light reflected oremitted from or transmitted through the consumable or consumablecomprising a sample.
 6. The system of claim 1, wherein the at least onesensor comprising: CCD, Photodiode, Photo multiplier tube, IR-NIRarrays, Focal Plane Array, InGaAs photodetector; VisGaAs photodetector,InSb photodetector, Quantum Well Infrared photodetector or combinationsthereof.
 7. The system of claim 1, wherein the sensor is an image sensorand captures an image of the reflected, emitted or transmitted lightfrom the sample, consumable or consumable comprising a sample.
 8. Thesystem of claim 7, wherein the captured images are standardized prior toanalysis by the software.
 9. The system of claim 7, whereinstandardization of images by the software comprises at least one stepof: selecting an area of interest for analysis; compensating forenvironmental artifacts; comparing each pixel value to an assignedthreshold value; identifying and enhancing objects; segmenting theimage.
 10. The system of claim 7, wherein the standardized image isanalyzed and results stored.
 11. The system of claim 1, furthercomprising a graphical user interface and one or more databases.
 12. Thesystem of claim 1, wherein said sensor comprising a charge-coupleddevice (CCD), focal plane array (FPA), near-infra red (NIR) sensor,NIR-VIS camera or combinations thereof.
 13. A method of detecting andanalyzing artifacts in a sample, consumable or consumable comprising asample, said method comprising: irradiating the sample with ultraviolet,visible, infra red or near infra red light; capturing an image of lightreflected from the sample or emitted or transmitted through the sample;processing and analyzing the image; and, detecting and analyzingartifacts in a sample, consumable or consumable containing a sample. 14.The method of claim 13, wherein the software standardizes the imageprior to analysis, said standardizing comprising at least one step of:selecting an area of interest for analysis; compensating forenvironmental artifacts; comparing each pixel value to an assignedthreshold value; identifying and enhancing objects; segmenting theimage.
 15. The method of claim 13, wherein analyzing comprises measuringat least one of: (a) volume of a liquid sample in wells (or microtubes)containing liquid sample, (b) detecting precipitate, objects orartifacts within microtiter plate wells, (c) classifying colored samplesin microtiter plate wells or microtubes; (d) determining contaminants;(e) air bubbles; (g) problems with a consumable.
 16. The method of claim15, wherein DMSO and water are quantitatively assessed whereinA_(x)=A_(water)+A_(DMSO)=e_(water)*b*C_(water)+e_(DMSO)*b*C_(DMSO) andA_(y)=A_(water)+A_(DMSO)=e_(water)*b*C_(water)=e_(DMSO)*b*C_(DMSO) whereconcentrations [A=absorbance, e=molar absorptivity constant,b=path-length and c=concentration].
 17. The method of claim 16, whereinx is >0 and y is >0.
 18. The method of claim 15, wherein the systemfurther comprises robotic integration, automation, semi automation orcombinations thereof.
 19. The method of claim 15, wherein remediation ofa contaminant comprises contacting a sample with a composition whichselectively removes the contaminant.